"The Nature of Selection", Price 1971
submitted by /u/gwern
[link] [comments]
-
Open
-
Open
-
Open
-
Open
-
Open
-
Open
Schmarzo and the Value·Nauts: The Journey from Data to Value
The journey from data and AI to value does not start with data or AI. Data and analytics are the new sources of value creation and competitive advantage in today’s economy. Some organizations have “cracked the” code in applying AI/ML to their data to uncover the customer, product, service, and operational predictive propensities (propensity to… Read More »Schmarzo and the Value·Nauts: The Journey from Data to Value
The post Schmarzo and the Value·Nauts: The Journey from Data to Value appeared first on Data Science Central.
( 23
min )
-
Open
On consequences of finetuning on data with highly discriminative features. (arXiv:2310.19537v2 [cs.LG] UPDATED)
In the era of transfer learning, training neural networks from scratch is
becoming obsolete. Transfer learning leverages prior knowledge for new tasks,
conserving computational resources. While its advantages are well-documented,
we uncover a notable drawback: networks tend to prioritize basic data patterns,
forsaking valuable pre-learned features. We term this behavior "feature
erosion" and analyze its impact on network performance and internal
representations.
( 2
min )
On some elusive aspects of databases hindering AI based discovery: A case study on superconducting materials. (arXiv:2311.09891v1 [cond-mat.other])
It stands to reason that the amount and the quality of big data is of key
importance for setting up accurate AI-driven models. Nonetheless, we believe
there are still critical roadblocks in the inherent generation of databases,
that are often underestimated and poorly discussed in the literature. In our
view, such issues can seriously hinder the AI-based discovery process, even
when high quality, sufficiently large and highly reputable data sources are
available. Here, considering superconducting and thermoelectric materials as
two representative case studies, we specifically discuss three aspects, namely
intrinsically biased sample selection, possible hidden variables, disparate
data age. Importantly, to our knowledge, we suggest and test a first strategy
capable of detecting and quantifying the presence of the intrinsic data bias.
( 2
min )
Fossil 2.0: Formal Certificate Synthesis for the Verification and Control of Dynamical Models. (arXiv:2311.09793v1 [eess.SY])
This paper presents Fossil 2.0, a new major release of a software tool for
the synthesis of certificates (e.g., Lyapunov and barrier functions) for
dynamical systems modelled as ordinary differential and difference equations.
Fossil 2.0 is much improved from its original release, including new
interfaces, a significantly expanded certificate portfolio, controller
synthesis and enhanced extensibility. We present these new features as part of
this tool paper. Fossil implements a counterexample-guided inductive synthesis
(CEGIS) loop ensuring the soundness of the method. Our tool uses neural
networks as templates to generate candidate functions, which are then formally
proven by an SMT solver acting as an assertion verifier. Improvements with
respect to the first release include a wider range of certificates, synthesis
of control laws, and support for discrete-time models.
( 2
min )
Rates of Convergence in Certain Native Spaces of Approximations used in Reinforcement Learning. (arXiv:2309.07383v3 [eess.SY] UPDATED)
This paper studies convergence rates for some value function approximations
that arise in a collection of reproducing kernel Hilbert spaces (RKHS)
$H(\Omega)$. By casting an optimal control problem in a specific class of
native spaces, strong rates of convergence are derived for the operator
equation that enables offline approximations that appear in policy iteration.
Explicit upper bounds on error in value function and controller approximations
are derived in terms of power function $\Pwr_{H,N}$ for the space of finite
dimensional approximants $H_N$ in the native space $H(\Omega)$. These bounds
are geometric in nature and refine some well-known, now classical results
concerning convergence of approximations of value functions.
( 2
min )
ICXML: An In-Context Learning Framework for Zero-Shot Extreme Multi-Label Classification. (arXiv:2311.09649v1 [cs.LG])
This paper focuses on the task of Extreme Multi-Label Classification (XMC)
whose goal is to predict multiple labels for each instance from an extremely
large label space. While existing research has primarily focused on fully
supervised XMC, real-world scenarios often lack complete supervision signals,
highlighting the importance of zero-shot settings. Given the large label space,
utilizing in-context learning approaches is not trivial. We address this issue
by introducing In-Context Extreme Multilabel Learning (ICXML), a two-stage
framework that cuts down the search space by generating a set of candidate
labels through incontext learning and then reranks them. Extensive experiments
suggest that ICXML advances the state of the art on two diverse public
benchmarks.
( 2
min )
Emerging Drug Interaction Prediction Enabled by Flow-based Graph Neural Network with Biomedical Network. (arXiv:2311.09261v1 [q-bio.QM])
Accurately predicting drug-drug interactions (DDI) for emerging drugs, which
offer possibilities for treating and alleviating diseases, with computational
methods can improve patient care and contribute to efficient drug development.
However, many existing computational methods require large amounts of known DDI
information, which is scarce for emerging drugs. In this paper, we propose
EmerGNN, a graph neural network (GNN) that can effectively predict interactions
for emerging drugs by leveraging the rich information in biomedical networks.
EmerGNN learns pairwise representations of drugs by extracting the paths
between drug pairs, propagating information from one drug to the other, and
incorporating the relevant biomedical concepts on the paths. The different
edges on the biomedical network are weighted to indicate the relevance for the
target DDI prediction. Overall, EmerGNN has higher accuracy than existing
approaches in predicting interactions for emerging drugs and can identify the
most relevant information on the biomedical network.
( 2
min )
Comparing Humans, GPT-4, and GPT-4V On Abstraction and Reasoning Tasks. (arXiv:2311.09247v1 [cs.AI])
We explore the abstract reasoning abilities of text-only and multimodal
versions of GPT-4, using the ConceptARC benchmark [10], which is designed to
evaluate robust understanding and reasoning with core-knowledge concepts. We
extend the work of Moskvichev et al. [10] by evaluating GPT-4 on more detailed,
one-shot prompting (rather than simple, zero-shot prompts) with text versions
of ConceptARC tasks, and by evaluating GPT-4V, the multimodal version of GPT-4,
on zero- and one-shot prompts using image versions of the simplest tasks. Our
experimental results support the conclusion that neither version of GPT-4 has
developed robust abstraction abilities at humanlike levels.
( 2
min )
The Cybersecurity Crisis of Artificial Intelligence: Unrestrained Adoption and Natural Language-Based Attacks. (arXiv:2311.09224v1 [cs.CY])
The widespread integration of autoregressive-large language models (AR-LLMs),
such as ChatGPT, across established applications, like search engines, has
introduced critical vulnerabilities with uniquely scalable characteristics. In
this commentary, we analyse these vulnerabilities, their dependence on natural
language as a vector of attack, and their challenges to cybersecurity best
practices. We offer recommendations designed to mitigate these challenges.
( 2
min )
-
Open
Modelling daily mobility using mobile data traffic at fine spatiotemporal scale. (arXiv:2311.09683v1 [cs.LG])
We applied a data-driven approach that explores the usability of the NetMob
2023 dataset in modelling mobility patterns within an urban context. We
combined the data with a highly suitable external source, the ENACT dataset,
which provides a 1 km x 1km grid with estimates of the day and night population
across Europe. We developed three sets of XGBoost models that predict the
population in each 100m x 100m grid cell used in NetMob2023 based on the mobile
data traffic of the 68 online services covered in the dataset, using the ENACT
values as ground truth. The results suggest that the NetMob 2023 data can be
useful for the estimation of the day and night population and grid cell level
and can explain part of the dynamics of urban mobility.
( 2
min )
Open Problem: Learning with Variational Objectives on Measures. (arXiv:2306.11928v2 [stat.ML] UPDATED)
The theory of statistical learning has focused on variational objectives
expressed on functions. In this note, we discuss motivations to write similar
objectives on measures, in particular to discuss out-of-distribution
generalization and weakly-supervised learning. It raises a natural question:
can one cast usual statistical learning results to objectives expressed on
measures? Does the resulting construction lead to new algorithms of practical
interest?
( 2
min )
Comparison of Methods that Combine Multiple Randomized Trials to Estimate Heterogeneous Treatment Effects. (arXiv:2303.16299v2 [stat.ME] UPDATED)
Individualized treatment decisions can improve health outcomes, but using
data to make these decisions in a reliable, precise, and generalizable way is
challenging with a single dataset. Leveraging multiple randomized controlled
trials allows for the combination of datasets with unconfounded treatment
assignment to better estimate heterogeneous treatment effects. This paper
discusses several non-parametric approaches for estimating heterogeneous
treatment effects using data from multiple trials. We extend single-study
methods to a scenario with multiple trials and explore their performance
through a simulation study, with data generation scenarios that have differing
levels of cross-trial heterogeneity. The simulations demonstrate that methods
that directly allow for heterogeneity of the treatment effect across trials
perform better than methods that do not, and that the choice of single-study
method matters based on the functional form of the treatment effect. Finally,
we discuss which methods perform well in each setting and then apply them to
four randomized controlled trials to examine effect heterogeneity of treatments
for major depressive disorder.
( 2
min )
Score-based generative models learn manifold-like structures with constrained mixing. (arXiv:2311.09952v1 [stat.ML])
How do score-based generative models (SBMs) learn the data distribution
supported on a low-dimensional manifold? We investigate the score model of a
trained SBM through its linear approximations and subspaces spanned by local
feature vectors. During diffusion as the noise decreases, the local
dimensionality increases and becomes more varied between different sample
sequences. Importantly, we find that the learned vector field mixes samples by
a non-conservative field within the manifold, although it denoises with normal
projections as if there is an energy function in off-manifold directions. At
each noise level, the subspace spanned by the local features overlap with an
effective density function. These observations suggest that SBMs can flexibly
mix samples with the learned score field while carefully maintaining a
manifold-like structure of the data distribution.
( 2
min )
Categorical Distributions of Maximum Entropy under Marginal Constraints. (arXiv:2204.03406v2 [hep-th] UPDATED)
The estimation of categorical distributions under marginal constraints
summarizing some sample from a population in the most-generalizable way is key
for many machine-learning and data-driven approaches. We provide a
parameter-agnostic theoretical framework that enables this task ensuring (i)
that a categorical distribution of Maximum Entropy under marginal constraints
always exists and (ii) that it is unique. The procedure of iterative
proportional fitting (IPF) naturally estimates that distribution from any
consistent set of marginal constraints directly in the space of probabilities,
thus deductively identifying a least-biased characterization of the population.
The theoretical framework together with IPF leads to a holistic workflow that
enables modeling any class of categorical distributions solely using the
phenomenological information provided.
( 2
min )
Adversarial Examples Exist in Two-Layer ReLU Networks for Low Dimensional Linear Subspaces. (arXiv:2303.00783v2 [cs.LG] UPDATED)
Despite a great deal of research, it is still not well-understood why trained
neural networks are highly vulnerable to adversarial examples. In this work we
focus on two-layer neural networks trained using data which lie on a low
dimensional linear subspace. We show that standard gradient methods lead to
non-robust neural networks, namely, networks which have large gradients in
directions orthogonal to the data subspace, and are susceptible to small
adversarial $L_2$-perturbations in these directions. Moreover, we show that
decreasing the initialization scale of the training algorithm, or adding $L_2$
regularization, can make the trained network more robust to adversarial
perturbations orthogonal to the data.
( 2
min )
Urban traffic congestion control: a DeePC change. (arXiv:2311.09851v1 [eess.SY])
Urban traffic congestion remains a pressing challenge in our rapidly
expanding cities, despite the abundance of available data and the efforts of
policymakers. By leveraging behavioral system theory and data-driven control,
this paper exploits the DeePC algorithm in the context of urban traffic control
performed via dynamic traffic lights. To validate our approach, we consider a
high-fidelity case study using the state-of-the-art simulation software package
Simulation of Urban MObility (SUMO). Preliminary results indicate that DeePC
outperforms existing approaches across various key metrics, including travel
time and CO$_2$ emissions, demonstrating its potential for effective traffic
management
( 2
min )
Online Optimization for Network Resource Allocation and Comparison with Reinforcement Learning Techniques. (arXiv:2311.10023v1 [stat.ML])
We tackle in this paper an online network resource allocation problem with
job transfers. The network is composed of many servers connected by
communication links. The system operates in discrete time; at each time slot,
the administrator reserves resources at servers for future job requests, and a
cost is incurred for the reservations made. Then, after receptions, the jobs
may be transferred between the servers to best accommodate the demands. This
incurs an additional transport cost. Finally, if a job request cannot be
satisfied, there is a violation that engenders a cost to pay for the blocked
job. We propose a randomized online algorithm based on the exponentially
weighted method. We prove that our algorithm enjoys a sub-linear in time
regret, which indicates that the algorithm is adapting and learning from its
experiences and is becoming more efficient in its decision-making as it
accumulates more data. Moreover, we test the performance of our algorithm on
artificial data and compare it against a reinforcement learning method where we
show that our proposed method outperforms the latter.
( 2
min )
